Comments for MEDB 5501, Week 6

Assessing normality

Problems caused by non-normality
- Poor confidence intervals, hypothesis tests
  - Too much imprecision
  - Poor coverage probability
    - Especially for one tailed tests
- Inability to extrapolate
What about the Central Limit Theorem?

Sometimes, I think that researchers obsess too much about non-normality, but in fairness to them, it is an important issue. If your data does not follow a bell shaped curve, then several problems could happen.

First, you might see a greater degree of imprecision, reflected in very wide confidence intervals and loss of statistical power.

Second, you might have poor coverage probability. The 95% confidence interval might only reprent a 92% confidence level. The Type I error rate might be greater than 5%.

Third, and this point is not emphasized enough, is that non-normal data makes it difficult for you to extrapolate to future observations. Prediction of future events is a big part of Statistics. It is difficult even with normal data and becomes much more difficult with non-normal data.

Yes, you might say, but doesn’t the Central Limit Theorem help us out? Well, yes, if the sample size is large, but it only helps with assuring accurate confidence levels and good control of the Type I error rate. Non-normal data will still often produce intervals that are too wide and tests that have too little power.

How to handle non-normality

Ignore it
- Central Limit Theorem
Transform your data
Use alternatives
- Nonparametric tests (covered in a later module)
- Bootstrap (covered in a later module)
- Randomization tests (not covered in this class)

Yes, you might say, but doesn’t the Central Limit Theorem help us out? Well, yes, but it only helps with assuring accurate confidence levels and good control of the Type I error rate. Non-normal data will still often produce intervals that are too wide and tests that have too little power.

Still, there often is benefit in not worrying so much about non-normality. If your sample size is large and the deviations from normality are minor, don’t let this keep you up at night.

Even so, there are some benefits to addressing non-normality. Transformations, which I will cover in just a bit, can often help out tremendously. There are also alternatives to the simple tests using a sample mean: nonparametric tests and the bootstrap. I will cover those in a later module. There are also randomization tests, which I don’t think are covered. I’ve given a talk on randomization tests and I have a few webpages that talk about this topic.

Approaches to examine normality

Histogram, boxplot
Normal probability plot
- Normal correlation (not covered, not recommended)
Skewness, Kurtosis
Kolmogorov-Smirnow test (not recommended)
Shapiro-Wilk test (not recommended)

There are many ways to check normality. The easiest one is a simple histogram. There’s a problem with the histogram in that you might draw different conclusions depending on how many bars you use. But it still is commonly used and easily understood.

The boxplot can also help distinguish normal versus non-normal data. It doesn’t do so well for certain types of non-normality, such as bimodal distributions, but it is also easy to use.

I’ll describe the normal probability plot in a bit. It requires a lot more interpretation, and you need to get some experience with it before you can get the feel of it. But it avoids some of the problems with the histogram and boxplot.

Some researchers advocate the test of a correlation based off of the normal probability plot as a measure of normality. I do not recommend this test.

There are two summary statistics, skewness and kurtosis, that are also commonly used.

There are two tests that I also do not recommend, the Kolmogorov-Smirnow test and the Shapiro-Wilk test

Never use p-values to test normality

H0: Data comes from a normal population
Why is this bad?
- ASA recommendation against the use of p-values
- Too little power for small sample sizes
- Too much power for large sample sizes
- Ignore the type of non-normality

Normal histogram with n=100

Normal histogram with n=1,000

Normal histogram with n=10,000

Normal histogram with n=100,000

Normal histogram with n=1,000,000

Normal distribution (n=infinity)

Normal(2, 1)

Normal(-1, 1)

Normal(0, 2)

Normal(0, 0.5)

Skewed right

Here’s an example of a distribution that is skewed to the right. I have scaled this distribution so that the mean is zero and the standard deviation is one.

There are three key features for a skewed right distribution.

First, the chances of seeing an outlier on the low end (more than two standard deviations below the mean) is less for a skewed right distribution. Skewed right distributions almost never produce outliers on the low end.

Second, the chances of seeing an outlier on the high end (more than two standard deviations above the mean) is greater for a skewed right distribution.

Third, the probability of any value being less than the mean is a bit more than 50%. That’s because the outliers on the high end of this distribution tend to have an undue influence on the mean, making it bigger than the bulk of the data.

Right skewness is characterized by the tails of the distribution

Heavy right tail
- Greater tendency to produce extreme values on the right
Light left tail
- Lesser tendency to produce extreme values on the left
Right skewness is the most common type of non-normality

Normal probability plot

Compare data to evenly spaced percentiles of the normal distribution
Example with n=4
- Compare smallest value with $Z_{0.2}$
- Compare next value with $Z_{0.4}$
- Compare next value with $Z_{0.6}$
- Compare largest value with $Z_{0.8}$
No best definition for evenly spaced
- 12.5, 37.5, 62.5, 87.5, for example

Interpreting, heavy left tail

Interpreting, light left tail

Interpreting, heavy right tail

Interpreting, light right tail

Right skewed data

Left skewed data

Heavy tailed data

Light tailed data

Bimodal data

Normal data

Log transformation

If $a^b=c$, then $log_a(c)=b$
$log(a \times b)=log(a)+log(b)$
Three commonly used bases
- log base 10: ($log_{10}$)
- log base 2: ($log_2$)
- natural log, log base e: ($ln$)
Important! SPSS uses lg10, not log10.

Where the log transformation is routine

Log units are common in science
- Richter scale (1 unit equals 10 fold change)
- Decibel (20 units equals 10 fold change)

Why use the log function

Stretches small values
Squeezes large values
Possible benefits
- Removing skew
- Eliminating outliers
- Stabilizing variation
- Model simplification

When to use a log transformation

Data bounded below by zero
Data defined as a ratio
Max / Min > 3

There are several things that you should look for when deciding whether to use a log transformation.

Data that is bounded below by zero is the first thing to look for, especially when your data comes close to, but never equals zero. The lower bound will often cause skewness because outliers can still appear on the high end, but not on the low end.

You should especially consider using the log transformation for data that represents ratios. Ratios are usually bounded below by zero. There is also an asymmetry in ratios. Ratios where then numerator is less than the denominator are confined to be between 0 and 1. When the numerator is larger than the denominator, the ratio can roam freely between 1 and infinity.

The log transformation works for measures other than ratios, so you should still consider it for other types of outcomes.

There’s an informal rule about the log transformation, though, that you should consider. Use it only when you have a wide spread of data, where the largest value in your data is more than three times as large as the smallest values in your data. If your data ranges from 200 to 300, the ratio of 1.5 indicates that you are unlikely to see much change with the log transformation.

Squeezing

Stretching

Skewness

Outliers

Unequal variation

Many statistical procedures require that all of your subject groups have comparable variation. If you data has unequal variation, then the some of your tests and confidence intervals may be invalid. A log transformation can help with certain types of unequal variation.

A common pattern of unequal variation is when the groups with the large means also tend to have large standard deviations. Consider housing prices in several different neighborhoods. In one part of town, houses might be cheap, and sell for 60 to 80 thousand dollars. In a different neighborhood, houses might sell for 120 to 180 thousand dollars. And in the snooty part of town, houses might sell for 400 to 600 thousand dollars. Notice that as the neighborhoods got more expensive, the range of prices got wider. This is an example of data where groups with large means tend to have large standard deviations.

With this pattern of variation, the log transformation can equalize the variation. The log transformation will squeeze the groups with the larger standard deviations more than it will squeeze the groups with the smaller standard deviations. The log transformation is especially effective when the size of a group’s standard deviation is directly proportional to the size of its mean.

Multiplicative models

Additive model
- Catalogs +1,000 causes sales +$5,000
Multiplicative model
- Rain + 1 inch causes pollen * 0.5
Multiplicative models are tricky
Log converts it to an additive model

There are two common statistical models, additive and multiplicative.

An additive model assumes that factors that change your outcome measure, change it by addition or subtraction. An example of an additive model would when we increase the number of mail order catalogs sent out by 1,000, and that adds an extra $5,000 in sales.

A multiplicative model assumes that factors that change your outcome measure, change it by multiplication or division. An example of a multiplicative model would be when an inch of rain takes half of the pollen out of the air.

In an additive model, the changes that we see are the same size, regardless of whether we are on the high end or the low end of the scale. Extra catalogs add the same amount to our sales regardless of whether our sales are big or small. In a multiplicative model, the changes we see are bigger at the high end of the scale than at the low end. An inch of rain takes a lot of pollen out on a high pollen day but proportionately less pollen out on a low pollen day.

If you remember your high school algebra, you’ll recall that the logarithm of a product is equal to the sum of the logarithms.

Therefore, a logarithm converts multiplication/division into addition/subtraction. Another way to think about this in a multiplicative model, large values imply large changes and small values imply small changes. The stretching and squeezing of the logarithm levels out the changes.

Example: Metabolic ratio

The DM/DX ratio is a measure of how rapidly the body metabolizes certain types of medication. A patient is given a dose of dextrometorphan (DM), a common cough medication. The patients urine is collected for four hours, and the concentrations of DM and DX (a metabolite of dextrometorphan) are measured. The ratio of DM concentration to DX is a measure of how well the CYD 2D6 metabolic pathway functions. A ratio less than 0.3 indicates normal metabolism; larger ratios indicate slow metabolism.

Genetics can influence CYP 2D6 metabolism. In this set of 206 patients, we have 15 with no functional alleles and 191 with one or more functional alleles.

The DM/DX ratio is a good candidate for a log transformation since it is bounded below by zero. It is also obviously a ratio. The standard deviation for this data (0.4) is much larger than the mean (0.1).

Finally, the largest value is several orders of magnitude bigger than the smallest value.

Skewness

Outliers

Unequal variation

Algebra formula for a straight line

$Y=mx+b$
$m = \Delta y / \Delta x$
m = slope
b = y-intercept

Linear regression interpretation of a straight line

The slope represents the estimated average change in Y when X increases by one unit.
The intercept represents the estimated average value of Y when X equals zero.

First regression example with interpretation

Output from SPSS

Interpretation when X is categorical

Code X as 0-1
Intercept: Estimated average value of Y for the “0” category.
Slope: Estimated average change in Y when category changes from “0” to “1”.

Second regression example with interpretation

SPSS output

Alternate coding

Figure 5. Scatterplot with alternate ordering of treatment

Comments for MEDB 5501, Week 6

Assessing normality

How to handle non-normality

Approaches to examine normality

Never use p-values to test normality

Normal histogram with n=100

Normal histogram with n=1,000

Normal histogram with n=10,000

Normal histogram with n=100,000

Normal histogram with n=1,000,000

Normal distribution (n=infinity)

Normal(2, 1)

Normal(-1, 1)

Normal(0, 2)

Normal(0, 0.5)

Skewed right

Right skewness is characterized by the tails of the distribution

Normal probability plot

Interpreting, heavy left tail

Interpreting, light left tail

Interpreting, heavy right tail

Interpreting, light right tail

Right skewed data

Left skewed data

Heavy tailed data

Light tailed data

Bimodal data

Normal data

Log transformation

Where the log transformation is routine

Why use the log function

When to use a log transformation

Squeezing

Stretching

Skewness

Outliers

Unequal variation

Multiplicative models

Example: Metabolic ratio

Skewness

Outliers

Unequal variation

Further reading

Algebra formula for a straight line

Linear regression interpretation of a straight line

First regression example with interpretation

Output from SPSS

Interpretation when X is categorical

Second regression example with interpretation

SPSS output

Alternate coding

SPSS output